1. Field of the Invention
The present invention relates to three-dimensional graphics and animation, and more particularly, to a motion tracking system that enables capture of facial and eye motion of a performer without the use of cameras for use in producing a computer graphics animation.
2. Description of Related Art
Motion capture systems are used to capture the movement of a real object and map it onto a computer generated object. Such systems are often used in the production of motion pictures and video games for creating a digital representation of a person that is used as source data to create a computer graphics (CG) animation. In a typical system, a performer wears a suit having markers attached at various locations (e.g., having small reflective markers attached to the body and limbs) and digital cameras record the movement of the performer from different angles while illuminating the markers. The system then analyzes the images to determine the locations (e.g., as spatial coordinates) and orientation of the markers on the performer's suit in each frame. By tracking the locations of the markers, the system creates a spatial representation of the markers over time and builds a digital representation of the performer in motion. The motion is then applied to a digital model, which may then be textured and rendered to produce a complete CG representation of the actor and/or performance. This technique has been used by special effects companies to produce incredibly realistic animations in many popular movies.
Motion capture systems are also used to track the motion of facial features of an actor to create a representation of the actor's facial motion and expression (e.g., laughing, crying, smiling, etc.). As with body motion capture, markers are attached to the actor's face and cameras record the actor's expressions. Since facial movement involves relatively small muscles in comparison to the larger muscles involved in body movement, the facial markers are typically much smaller than the corresponding body markers, and the cameras typically have higher resolution than cameras usually used for body motion capture. The cameras are typically aligned in a common plane with physical movement of the actor restricted to keep the cameras focused on the actor's face. The facial motion capture system may be incorporated into a helmet or other implement that is physically attached to the actor so as to uniformly illuminate the facial markers and minimize the degree of relative movement between the camera and face.
An advantage of motion capture systems over traditional animation techniques, such as keyframing, is the capability of real-time visualization. The production team can review the spatial representation of the performer's motion in real-time or near real-time, enabling the actor to alter the physical performance in order to capture optimal data. Moreover, motion capture systems detect subtle nuances of physical movement that cannot be easily reproduced using other animation techniques, thereby yielding data that more accurately reflects natural movement. As a result, animation created using source material that was collected using a motion capture system will exhibit a more lifelike appearance.
Notwithstanding these advantages of motion capture systems, a drawback of conventional motion capture systems is that they cannot capture eye motion. Since the markers cannot be affixed to the performer's eyes, the eye movement is not detected by the motion capture cameras. This eye movement must then be added during the subsequent CG animation process. In addition to making the animation process more cumbersome, the resulting animation product is less realistic since it may not include subtle eye movement that occurs during a performance.
Another drawback of conventional motion capture systems that rely upon cameras is that motion data of a performer may be occluded by interference with other objects, such as props or other actors. Specifically, if a portion of the body or facial markers is blocked from the field of view of the digital cameras, then data concerning that body or facial portion is not collected. This results in an occlusion or hole in the motion data. While the occlusion can be filled in later during post-production using conventional computer graphics techniques, the fill data lacks the quality of the actual motion data, resulting in a defect of the animation that may be discernable to the viewing audience. To avoid this problem, conventional motion capture systems limit the number of objects that can be captured at one time, e.g., to a single performer. This also tends to make the motion data appear less realistic, since the quality of a performer's performance often depends upon interaction with other actors and objects. Moreover, it is difficult to combine these separate performances together in a manner that appears natural.
Outside of the entertainment industry, there are many other circumstances in which it would be desirable to capture or track facial muscle and/or eye movement without reliance upon optical cameras. For example, automatic speech recognition devices, access control systems, electronic storage and retrieval systems for personal profiles and medical/dental screening systems could make use of such analysis techniques. Speech recognition systems that utilize analyses of facial features may find wide application in noisy environments where it is difficult to utilize acoustic speech recognition alone, e.g., in a military aircraft or in a factory. Each of these potential applications presently lack an effective means for accurately translating an individual's facial features into useful electronic data. This is particularly problematic where the individual is continually changing facial orientation with respect to the detection equipment.
Accordingly, it would be desirable to provide a motion tracking system that overcomes these and other drawbacks of the prior art. More specifically, it would be desirable to provide a motion tracking system that enables faithful capture of subtle facial and eye motion of a performer without the use of cameras.